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Forum 


This practical paper discusses the effect of 
explicit instruction to raise Japanese EFL 
learners' pragmatic awareness using online 
discourse completion tasks. The five-part tasks 
developed by the authors use American TV 
drama scenes depicting particular speech acts 
and include explicit instruction in these speech 
acts. 46 Japanese EFL college students enrolled 
in an English grammar course received explicit 
instruction and completed the tasks in five con¬ 
secutive classes using an e-portfolio system 
that allows them to share their learning out¬ 
comes. Their pragmatic awareness was mea¬ 
sured by pre- and post-instruction pragmatic 
judgment tasks. The results showed that the 
participants' pragmatic awareness developed 
significantly after the instruction. The learn¬ 
ers evaluated the awareness raising activities 
as effective; however, their learning outcomes 
reveal challenges in developing grammatical 
and lexical knowledge together with pragmatic 
knowledge. This paper will discuss the effects 
of the instruction , the participants'perception 
of the usefulness of the tasks , and their actual 
responses to the tasks to further identify the 
challenges that Japanese EFL learners should 
meet to develop well-balanced communicative 
competence in the language. 

Keywords: Interlanguage pragmatics, prag¬ 
matic awareness, CALL, e-portfolio 

1. Introduction 

Pragmatic competence is defined as "the 
knowledge that influences and constrains 
speakers' choices regarding use of language 
in socially appropriate ways" (LoCastro, 2012, 
p. 307) and is regarded as a major compo¬ 
nent of an influential language knowledge 143 
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model (Bachman & Palmer, 2010). It is also related to the overall objective of English in 
the Course of Study (MEXT, 2009) "to develop students' communication abilities such as... 
appropriately conveying information" (p. 1). L2 pragmatics or interlanguage pragmatics, 
"the study of how L2 learners learn how-to-say-what-to-whom-when" (Bardovi-Harlig, 2013, 
p. 216), has been widely studied for these past three decades as it has been claimed that L2 
learners often develop grammatical competence in the absence of concomitant pragmatic 
competence (Bardovi-Harlig & Dornyei, 1998, p. 233). 

Previous studies have shown that pragmatic competence is both teachable and learnable 
for both EFL and ESL learners (Martinez-Flor & Fukuya, 2005; Olshtain & Cohen, 1990; 
Wigglesworth & Yates, 2007). There are also studies exploring the effect of instruction on 
Japanese EFL learners' pragmatic competence (Kondo, 2008; Takahashi, 2001; Takimoto, 
2008). In general, these studies found that explicit instruction had a stronger effect on 
learners' pragmatic competence than implicit instruction, for Japanese EFL learners as well 
as for learners in different contexts. 

While previous studies support the leamability of pragmatic competence, it is also 
claimed that there is "a noticeable gap between what research in pragmatics has found and 
how language is generally taught today" (Cohen, 2012, p. 33); in other words, pragmatics 
instruction in the language classroom is not taking place in an appropriate timing or man¬ 
ner. Sykes (2009) lists eight reasons to explain why this lack in language classrooms occurs: 

1. limited theoretical support in course design and material creation, 

2. lack of authentic curricular materials and appropriate input, 

3. lack of instructor expertise, 

4. a focus on microlevel features instead of macrolevel competence, 

5. limited time available in the L2 classroom, 

6. individual personality differences and sensitivity to certain factors influencing the 
interaction, 

7. assessment and feedback challenges, and 

8. immense dialectical, social and individual variation (pp. 203-204). 

Pragmatic awareness, "the conscious, reflective, explicit knowledge about pragmatics" 
(Alcon & Jorda, 2008, p. 193), is crucial for the intercultural speaker to be competent at 
the pragmatic level (Cenoz, 2007) and plays an important role in developing pragmatic 
competence (Tagashira, Yamato, & Isoda, 2011). There are numerous studies that investi¬ 
gated relationships between pragmatic awareness and other related factors. Bardovi-Harlig 
and Dornyei (1998) were some of the first researchers to examine the effects of learning 
environment on the development of pragmatic awareness along with grammatical aware¬ 
ness. Researchers have followed this line of research, investigating the development of L2 
pragmatic and grammatical awareness in relation to various factors such as learning envi¬ 
ronment (Niezgoda & Rover, 2001; Schauer, 2006), proficiency (Niezgoda & Rover, 2001; Xu, 
Case, & Wang, 2009) and motivational profile (Tagashira et ai, 2011; Yamato, Tagashira, & 
Isoda, 2013). Research has revealed that it is difficult for learners in foreign language learn¬ 
ing settings to develop pragmatic awareness eguivalent to that of ESL learners (Schauer, 
2006). 

In order to fill the gap between what research has found regarding the importance of 
developing pragmatic competence and the lack of its teaching in Japanese EFL settings, 
this practical study tackles the issue of raising pragmatic awareness with online discourse 
144 completion tasks using authentic video materials combined with explicit instruction by 
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classroom instructors and the sharing of learning outcomes among peers on an e-portfolio 
system. The research guestions are as follows: 

RQi: Does explicit instruction using online discourse completion tasks raise Japanese EFL 
learners' pragmatic awareness? 

RQ2: How do Japanese EFL learners respond to online discourse completion tasks 
combined with explicit instruction? 

RQ3: How do Japanese EFL learners perceive the effect of explicit instruction of 
pragmatic awareness? 

2. Method 

2.1 Participants 

The participants were 46 Japanese university freshmen majoring in English language and 
its cultures who were enrolled in a 15-week elective English grammar course to learn basic 
grammatical knowledge and skills. Each week, a ninety-minute class was conducted in a 
computer room allowing each participant to access online materials and tests. Demographic 
data of the same participants showed that there was no one who had spent and/or studied 
more than six months in an English speaking country. Judging from the passing marks for 
admission in the National Center Test for University Admissions, a standardized university 
entrance exam for public and private colleges, the participants' department is considered 
to be at an average academic level in Japan. 

2.2 Materials 

2.2.1 Online discourse completion task. To provide video materials and to share the learn¬ 
ing outcomes among the participants, the authors adopted Lexinote (Tanaka, Yonesaka, 
Ueno & Ohnishi, 2015), an e-portfolio system, as the platform for the online discourse 
completions tasks. Lexinote allows learners to record and save the target vocabulary items 
they encounter online, to search for them in online dictionaries, to practice them in sev¬ 
eral ways including written and oral rehearsals according to word familiarity (i.e., level of 
understanding of each lexical item), and to share their own learning experiences - in this 
study, responses to a discourse completion task - with peers. 

Each online discourse completion task consists of one screen with five parts. One part 
displays a discourse completion task that reguires the participants to complete a speech 
act in writing such as refusal, response to compliment, and offer. Next is a video clip from 
an American TV drama series, The Big Bang Theory (CBS Corporation), in which the target 
speech act is depicted. Underneath the video clip is the input held for the participants to 
provide their own response to the task shown in the first part. The speech act performances 
included not only successful speech acts but also problematic ones in order to explain why 
certain types of performance may cause communication breakdowns. There is also the script 
of the video clip followed by an Li explanation of typical and/or problematic speech acts 
and strategies. Finally, the participants' responses are shown in a display to allow them to 
share learning outcomes in class. Figure 1 shows an example screenshot of a refusal task 
for this study. 
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Figure 1 . An example screenshot of Lexinote, e-portfolio. 


2.2.2 Pragmatic judgment task. To measure the participants 7 pragmatic awareness, the 
authors adopted the pragmatic and grammatical judgment task developed by Bardovi- 
Harlig and Dornyei (1998). The task is intended to reveal learners 7 awareness in two steps; 
first, learners identify whether an item is erroneous, either pragmatically or grammatically, 
or not; and second, if erroneous, learners judge how severe the error is on a 6-point Likert 
scale. The task contains 20 scenarios that include four speech acts: reguests, apologies, 
suggestions, and refusals. 

In order to avoid any practice effect of using the same task for both pre- and post-tests, 
the authors made two versions of the task using parallel sentences to the original task. 
Another revision was made for the number of items in the task to yield a wider spread in the 
scores. The original task consisted of eight grammatically incorrect items, eight pragmati¬ 
cally inappropriate items and four non-erroneous items. By adding three grammatically 
incorrect items, three pragmatically inappropriate items, and one non-erroneous item for 
each test, the authors developed two revised versions of the pragmatic and grammatical 
judgment task with a total of 27 items each. The newly added items were first reviewed by 
four English native speakers (one American and three Canadians), and then revised and 
reviewed by three native English speaking EFL professionals in Japan, (one American, one 
Canadian, and one Australian). The final versions of the pragmatic and grammatical judg¬ 
ment tasks consisted of 11 grammatically incorrect items, 11 pragmatically inappropriate 
items and five non-erroneous items each. 

Following up on other researchers 7 use of the original task (Tagashira et at, 2011), the 
authors also changed the choices to make it possible to see which types of errors the 
participants actually identify. The following is an example of pragmatic errors in the pre¬ 
test (Set A) and the post-test (Set B). Task instructions were translated from English into 
146 Japanese by the authors. 
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An example question of the revised pragmatic and grammatical judgment task (prag¬ 
matic error): Set A 


It's Anna's day to give her talk in class , but sbe's not ready. 

Teacher: Thank you Peter, that was very interesting. Anna, its your turn to give your talk. 
(!) Anna: I can't do it today, but I will do it next week. 

The last part (!) is ... 

1. grammatically incorrect 

2. inappropriate for the situation 

3. grammatically correct and appropriate for the situation 
4.1 don't know 

If there is a problem, how bad do you think it is? 

Not bad at all_:_:_:_:_: V ery bad 


An example question of the revised pragmatic and grammatical judgment task (prag¬ 
matic error): Set B 


It's Anna's day to give her presentation in class , but sbe's not ready. 

Teacher: Thank you Daniel, that was really amazing. Anna, it's your turn to give your 
presentation. 

(!) Anna: I can't do it today, but I will do it tomorrow. 

The last part (!) is ... 

1. grammatically incorrect 

2. inappropriate for the situation 

3. grammatically correct and appropriate for the situation 
4.1 don't know 

If there is a problem, how bad do you think it is? 

Not bad at all_:_:_:_:_: V ery bad 


2.2.3 Questionnaire. In order to see how Japanese EFL learners perceive the effect of the 
explicit instruction, i.e., research question 3, the authors used a simple two-item question¬ 
naire. Item 1 uses a 6-point Likert scale from "1 .1 do not agree at all." to “ 6 .1 strongly agree." 
Item 2 is an open-ended question asking the participants to state their perception of the 
online completion tasks. 

1. Class assignments using Lexinote are useful to learn appropriate use of English. 

2. What do you think of the discourse completion task using Lexinote? 

2.3 Procedure 

During a 15-week format semester, the pre-test was conducted in Week 3, five weeks before 
the instruction started, in order to assess the participants' initial pragmatic awareness as 
well as to avoid a practice effect by delivering two similar tests within a short period. The 
instruction started in Week 9 and lasted until Week 13. The post-test was conducted in Week 
14. Both the pre- and post-tests were delivered via a learning management system, Glexa 
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(Version2 Inc., 2013), at the participants' university. Table 1 shows the schedule of the tests 
and the instruction of speech acts. 


Table 1 : The tests and speech act instruction schedule 


Week 

Test and instruction 

Week 3 

Pre-test (Pragmatic and grammatical judgment task Set A) 

Week 6 

Lexinote system hands-on practice (continued to Week 8) 

Week 9 

Task 1: Offering 

Week 10 

Task 2: Refusing an Offer 

Week 11 

Task 3: Requesting 

Week 12 

Task 4: Responding to a Compliment 

Week 13 

Task 5: Refusing a Request 

Week 14 

Post-test (Pragmatic and grammatical judgment task Set B) / Questionnaire 


Each instruction session followed the procedures below. 

1. Participants watch a video clip depicting a particular speech act. (3 min.) 

2. Participants listen to and read the teacher's Li explanation of pragmatic strategies and 
cross-cultural differences. (10 min.) 

3. Participants respond to the online written discourse completion task. (3-5 min.) 

4. Participants share their responses on the screen and the instructor comments on their 
work. (5-10 min.) 

To raise participants' pragmatic awareness, throughout all the procedures, the instructors 
of the course, also the authors of this study, encouraged the participants to think of their 
own way of performing speech acts in English. Before showing a video clip, the instructors 
drew the participants' attention to the targeted speech act and to whether it was an ideal 
one or a failure, by explaining the background of the scene and the characters. After show¬ 
ing the video, the instructors explained various possible speech act strategies (e.g., Beebe 
et al., 1990) so that the participants could consider which strategies they should take given 
the context. Following the explanation, the participants responded to the online written dis¬ 
course completion task. Since the responses sent to the system were automatically updated 
every 10 seconds, the participants were able to see other participants' responses immedi¬ 
ately after they were sent to the system. After all the participants sent their responses, the 
instructors gave feedback regarding the strategies they had used and any grammatical 
mistakes they had made. According to the feedback they received, the participants deleted 
their initial responses and sent new responses if they wanted. 

3. Results and discussion 

3.1 Pragmatic awareness 

Only the pragmatic items in the judgment task were used in this analysis since the focus of 
this practical study is on pragmatic awareness. Because a total of eight participants were 
absent at either the pre-test or the post-test session, the data from 38 participants were used 
148 for the analysis. Table 2 shows the descriptive statistics of the scores of the pre-test and the 













Tanaka & Oki: Raising pragmatic awareness using online discourse completion tasks 


post text. The results showed the effects of explicit instruction on participants' pragmatic 
awareness. Mean scores, with a maximum of 11 points, increased from 5.18 C SD = 2.42) in 
the pre-test to 6.13 (SD = 2.27) in the post-test; thus, the error recognition rate increased 
from 47% to 56% after instruction. A paired t-test between the pre- and post-test scores 
showed that there was a significant difference in the pragmatic error recognition rate 
between pre- and post-instruction with a medium effect size (t (37) = 2.71, p = .01, r = .41). 
The 95% confidence interval for the difference was from -1.66 to -0.24. Figure 2 shows the 
boxplots of the pre-test and the post-test. 

Table 2 : Descriptive statistics of the pre-test and the post-test (n = 38 ) 



M 

Min. 

Max. 

SD 

95% Cl 

Pre-test 

5.18 

1 

10 

2.42 

[4.39, 5.98] 

Post-test 

6.13 

1 

11 

2.27 

[5.39, 6.88] 



Pre-test Post-test 

Figure 2 . Boxplots of the pre-test and the post-test results. 

As for severity rating, i.e., how severely the participants evaluated the pragmatic errors, the 
scores increased slightly from the pre-test (M = 4.61, SD = 0.52) to the post-test (M = 4.79, 
SD = 0.73). Because it was not always the case that the same participants recognized the 
same types of errors in both tests, and also because the number of the participants who 
successfully recognized each error varied, the authors did not proceed with any further 
statistic analysis for severity rating. Whether classroom instruction affects learners' evalu¬ 
ation of pragmatic errors should be further investigated in a rigidly designed experimental 
study, because not only classroom instructions affect the evaluation of appropriateness of 
language use but also other factors such as individual and socio-cultural differences do so. 
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3.2 Learners' response to online discourse completion task 

From among the four speech acts instructed in the study, this section focuses on one speech 
act, refusal, to see how the participants actually responded to the online discourse comple¬ 
tion task and explicit instruction. The speech act of refusal has been claimed to be a major 
cross-cultural "sticking point" for non-native speakers and is interesting from a sociolin- 
guistic point of view because of its complexity and its various forms (Beebe, Takahashi, & 
Uliss-Weltz, 1990). 

Figure 3 shows the online discourse completion task of refusal to a reguest for Week 
13 which reguires the participants to refuse a friend's reguest to borrow their notebook. 
According to Beebe et at. (1990), the discourse of refusal by American English speakers is 
generally divided into three stages: give a positive opinion, make a statement of regret, and 
give an excuse. The following are some typical examples from the participants' responses. 
Any grammatical and/or vocabulary mistakes are left uncorrected. 


One of your friends missed an important class and wants to borrow your notebook , but you 
have to refuse your friend's request for some reason. Wbat would you say ? 

Your friend: Hi, do you have a minute? 

You: Sure. 

Your friend: I actually missed the economics class yesterday. I got up late, and I couldn't 
make it. But we have the midterm next week, right? So, I was wondering if I could bor¬ 
row your notebook, I mean, just the last part is fine. 

You: Well, (_). 

Your friend: Oh, OK. Don't worry. I'll ask some other friends. Thanks! 


Figure 3. Discourse completion task for Week 13: Refusal to request. 


Example 1. 

Welt, that's too bad. I want lend you my notebook , but I'm sorry. I bare to study bard today. 

(Student A) 


Example 2. 

Well I'm sorry. I wish I could that, but I want to study at borne with it because of the test next 
week. So I want you to ask for others. 

(Student B) 


Example 3. 

Well I know bowyoufeel. I was wondering if I could borrow my note , but I'm going to borrow 
my note other people. You should bcwe told me earlier. 

(Student C) 


The first example begins with empathy, "Well, that's too bad," and then a positive opinion, 
"I want (to) lend you my notebook," followed by an apology, "I'm sorry", and ending with 
an excuse, "I have to study hard today." Though there is a grammatical mistake. Student 
A clearly tried to follow the strategies as instructed. The second example begins with an 
150 apology, "I'm sorry" and then expresses regret "I wish I could [do] that," provides an excuse 
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"I want to study at home," and makes a suggestion "I want you to ask (for) others." The use 
of "I'm sorry" here could be seen as a transfer of Li use of “gomen(ne)” in Japanese, showing 
that Student B intends to be polite to the hearer, but this politeness is contradicted by the 
use of the impolite expression T want you to" at the end. The third example begins with 
a statement of empathy, "I know how you feel," though it is difficult to judge whether the 
expression is appropriate or not in this situation considering individual differences in the 
perception of relationships with the friend. After that. Student C misuses the verb "borrow" 
instead of "lend" or "let you use" and uses the modal "could" instead of "should" or "would 
need to". Another possible inappropriate use is "You should have told me earlier", although 
it depends on how close the friends are. 

Overall, these examples demonstrate that the participants tried to some extent to fol¬ 
low the strategies or the discourse of refusal, as instructed in the class, ffowever, many of 
the overall responses contain grammatical and/or word usage errors that affect the whole 
discourse and the hearer's impression of the speaker. Also, it should be noted that these 
responses were collected immediately after each instructional session, but whether the 
participants actually acguired a particular pragmatic strategy would have to be further 
investigated using a delayed post-test, which was not included in this study. 

3.3 Learners' perception of explicit instruction of pragmatic awareness 

Regarding guestionnaire Item 1, the usefulness of Lexinote in learning the appropriate 
use of English, the results showed that the participants perceived the online task to be 
useful to some extent (M = 4.70, SD = 0.90). Although it is difficult to tell whether this 
mean score is meaningful or not without additional survey items, only three participants 
responded "I don't agree" or "If I have to choose, I don't agree.", and no participants chose 
"I don't agree at all." 

The participants' responses to open-ended Item 2 asking what they thought of the dis¬ 
course completion task using Lexinote, showed a positive evaluation of the instruction 
and of the task, and also implied how these should be modified. Below are three excerpts 
from the three negative respondents, followed by three excerpts from the other 43 posi¬ 
tive respondents. The negative responses were related either to uncertainty about using 
Lexinote or to doubts of using it compared to face-to-face practice. The typical positive 
responses were related to the following three points: the effectiveness of observing conver¬ 
sation in a TV drama, reflections on their own language use, and the advantage of sharing 
learning products with peers. All the excerpts below were translated from Japanese into 
English by the authors. 

Excerpt 1. 

It was great to team bow I should use English grammar by watching the videos depicting 

actual conversation , but I was not so sure how to use Lexinote. 

(Student D) 


Excerpt 2. 

I didn't fully understand how to use the system (Lexinote), so I didn't feel its effectiveness for 
my learning. 

(Student E) 
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Excerpt 3. 

It was good that I teamed grammar mles by watching the videos, but I would rather practice 
the mtes in pairs than uploading sentences to Lexinote. 

(Student F) 


Excerpt 4. 

The advantages of doing this task are that I have fun learning English by watching an American 
TV drama, that I can see other students'work on the system, and that I can understand how 
we react differently from each other 

(Student G) 


Excerpt 5. 

I always had fun watching the videos, and I felt that I enjoyed learning by seeing other students' 
work on the system. Also the tasks give me a chance to think of how I should communicate 
with others in English■ 

(Student H) 


Excerpt 6. 

I felt it useful and effective to write my responses by thinking of each situation and of rela¬ 
tionships with whom I communicate. For example, I learned that there are various ways and 
expressions of apologies through the task. 

(Student I) 


4. Conclusion 

This practical study showed that the explicit instruction using discourse completion tasks 
had a significant impact on Japanese EFL learners' pragmatic awareness, and that the 
learners perceived the usefulness of the tasks in learning appropriate use of English. It 
also showed that the learners in this study attempted to construct their discourse using 
the strategies that were taught in class despite challenges in grammatical and word usage 
knowledge. 

The fact that learners have difficulties in both correctness and appropriateness of lan¬ 
guage use implies that Japanese EFL instructors should conduct classes that balance gram¬ 
matical and pragmatic aspects of language. It is claimed, however, that "textbooks still tend 
to provide only limited coverage of how language in context conveys messages directly 
and indirectly" (Cohen, 2012, p. 34). Classroom instructors' efforts must necessarily play a 
significant role in developing learners' well-balanced language competence. 

In order to further investigate the potential of explicit instruction on Japanese EFL learn¬ 
ers' L2 pragmatic awareness in a rigidly designed experimental study, the following limita¬ 
tions have to be solved. First, as the instruction of this study was done in only five classes 
using less than 30 minutes each, it is necessary to examine the effect of instruction over a 
longer period and to examine whether the effect will last over time in a pre-test, post-test 
and delayed post-test design. Second, though the authors carefully used parallel sentences 
in similar situations to the original judgment task by Bardovi-Harlig & Dornyei (1998), it 
is also necessary to confirm that the two re-designed awareness tasks in this study are at 
the same level of difficulty. Finally, in order to see whether learners actually benefit from 
the instruction, a future study should use an oral discourse completion task to elicit learn- 
152 ers' actual language use. Since this study employed a written online discourse completion 
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task where the participants could spend some time before they responded to the task, the 
results would have been different if oral discourse completion tasks had been used. By solv¬ 
ing these limitations, researchers and classroom instructors can further reveal the potential 
of explicit instruction on L2 learners' pragmatic awareness. 
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